Embodiments described herein generally relate to three-dimensional (3D) image data sets of a patient and in particular how to position anatomical landmarks therein.
In the medical field, three-dimensional (3D) image data sets, i.e. volume data sets, are collected by a variety of techniques—referred to as modalities in the field—including computer assisted tomography (CT), magnetic resonance (MR), single photon emission computed tomography (SPECT), ultrasound and positron emission tomography (PET). Volume data comprises a plurality of voxels arranged in a 3D grid. Each voxel has a voxel value associated with it. The voxel values represent measurements of a physical parameter. For example, in the case of CT scans, the voxel values represent the opacity of those voxels to X-rays, i.e. their X-ray stopping power. X-ray stopping power is measured in Hounsfield units (HUs) which is closely correlated with density (mass per unit volume).
Anatomical landmarks play an important role in labelling patient images. Conventionally, they are used as a point of reference by clinicians either to define a specific location for its own sake, or to provide a location used for a measurement such as a distance measurement or an angular measurement to or relative to another landmark.
Volume data sets are visualised with a visualization application, i.e. a computer program, to present two-dimensional (2D) images on a screen calculated from the volume, i.e. 3D, patient data set. The process of calculating two-dimensional (2D) images of 3D objects is referred to as volume rendering. Volume rendering can be applied to slab multi-planar reformatting (MPR), for example.
Accurate detection and classification of anatomical landmarks in volume data sets is useful for a rendering application, since it can enable user navigation of the patient image data and can be used to trigger contextually-dependent functionality and analysis. Anatomical landmarks can, for example, be used to provide context-based visualization options, such as choices of potentially suitable presets from among a large library of presets. [A preset defines a mapping between image values in an image data set and what shade pixels or voxels having those image values are given when displayed to a user—typically by selecting a particular combination of window level and window width known to be suitable for visualizing a particular tissue type]. Landmarks may also be used as an aid to segmentation or in conjunction with segmentation data.
Anatomical landmarks also allow comparison of different images and their registration without original data, such as between images from two different patients, or of the same patient taken at different times (e.g. movie frames taken a fraction of a second apart, or images taken weeks, months or years apart such as pre- and post-operative or -trauma).
Anatomical landmarks may also be used in their conventional way as an interpretation aid by a clinician using the visualization application.
However, the nature of patient data is that it is inherently dynamic and noisy. Moreover, the human body comes in a wide variety of shapes and forms, for example as dictated by age, so accurate positioning of landmarks in a patient image will always be a challenge.
It is therefore an important task of the visualization application to accurately place landmarks in an automated way.
A discriminative classification approach, such as provided by decision trees or decision forests, is one known automated method of placing anatomical landmarks in a patient image data set.
A discriminative classification approach to place anatomical landmarks will generate possible candidate locations independently of each other which can be a disadvantage as now described with an example.
FIG. 1 is an image of a CT image data set of the thorax and pelvic region of a patient. Superposed on the image are a number of landmark points. The “cross” landmark points are correct and are derived from manual placement, i.e. represent the ground truth. The “dot” landmark points are at locations placed by a classification forest method according to the prior art, and incorporate a mixture of minor and gross errors. Corresponding ground truth (cross) and modeled (dot) landmark points are linked with a straight line. Where no line is shown, the landmark does not appear in the ground truth set for this image. The classification approach has independently assigned the locations of the anatomical landmarks. However, according to anatomical knowledge, the correct configuration of the detected landmarks is violated. This is due to the absence of global awareness to the relative location of landmarks. Correctly identified landmarks are not able to assist in correcting errors in the placement of other landmarks.
In particular, the modelled (dot) landmark inside the oval in the right hand side of the figure is grossly misplaced compared to the ground truth (cross) landmark which is some distance above it, but the classification forest model is not able to use the (almost) correctly placed neighbouring landmarks as a cue to correct the grossly misplaced landmark.
Further, the cluster of modelled (dot) landmarks circled near the bottom right of the image are in fact head landmarks. The placement of head landmarks by the classification forest takes place, because the decision forest is not spatially aware of neighbouring detected landmarks, so their position will not be corrected despite the obvious misclassification error. This feature is general to any discriminative detection approach including other types of decision forest.